Smart Wand Device

ABSTRACT

Devices, systems, and methods for collecting data with a smart wand device held in a user&#39;s hand. The smart wand device includes an elongated barrel housing, a position sensor, a transceiver, and a processor. The elongate barrel housing extends between a tip and an end cap of the smart wand device. The position sensor is configured to determine changes in position of a reference point of the smart wand device. The smart wand device is configured to collect sensor data from the position sensor. The sensor data measures movements of a gesture made with the tip in midair by the user holding the elongate barrel housing. The gesture is made without contact between the tip and a surface. The sensor data is transmitted to a remote computing device for conversion into a two-dimensional rendering corresponding to the gesture made with the tip in midair by the user.

RELATED APPLICATIONS

This application is a continuation of U.S. Non-Provisional applicationSer. No. 15/625,511, titled “Smart Wand Device” filed Jun. 16, 2017, theentire content of which is hereby incorporated by reference for allpurposes.

BACKGROUND

Note taking has evolved from using writing instruments on a surface,like pen and paper, to modern computing devices that receive andcommunicate numerous kinds of data electronically. Smart phones andtablet computers are powerful mobile computing devices that have enabledmany new kinds of note taking and communications, but generally requiredirect interaction with a touch-screen or buttons on the device. Eventhe act of unlocking mobile computing devices requires direct physicalinteractions, such as entering a password or using a fingerprint scanneron the device. Such direct physical interactions can be inconvenient,particularly when the mobile computing device is not readily accessible(e.g., out of reach or stored in a bag or pocket).

Electronic writing instruments, like smart pens, do allow a user towrite on a surface in a traditional manner, while recording the surfaceinteractions on a connected computing device. However, such electronicwriting instruments generally require a special writing surface and/orare not able to unlock the smart phone. It would be desirable to have asmall accessory that could be used to unlock, activate, enter notationsinto, and even interact with the mobile computing device without theneed for a special writing surface.

SUMMARY

In some embodiments, methods, devices, and systems may collect data witha smart wand device held in a user's hand. The smart wand deviceincludes an elongate barrel housing, a position sensor, a transceiver,and a processor. The elongate barrel housing may extend between a tipand an end cap of the smart wand device. A portion of the elongatebarrel housing may be sized to be held by a user. The position sensormay be secured to the elongate barrel housing and configured todetermine changes in position of a reference point of the smart wanddevice. The processor may be disposed within the elongate barrel housingand coupled to at least the position sensor and the transceiver. Theprocessor may be configured with processor-executable instructions toperform operations including receiving sensor data from the positionsensor. The sensor data may measure movements of a gesture made with thetip in midair by the user holding the elongate barrel housing. Thegesture may be made without contact between the tip and a surface. Thesensor data may be transmitted, using the transceiver, to a remotecomputing device for conversion into a two-dimensional renderingcorresponding to the gesture made with the tip in midair by the user.

Various embodiments may include a fingerprint scanner disposed on anouter portion of the elongate barrel housing and coupled to theprocessor. The fingerprint scanner may be configured to detectfingerprint features of the user in response to the user engaging afinger on the fingerprint scanner while the user is holding the elongatebarrel. The processor may be configured with processor-executableinstructions to receive data representing the fingerprint features. Theprocessor may also transmit the data representing the fingerprintfeatures, using the transceiver, to the remote computing device. Inaddition, the processor may receive an indication from the remotecomputing device that the data representing the fingerprint features wasaccepted for unlocking the remote computing device. The reference pointused to determine changes in position may correspond to a nib of thesmart wand device.

Various embodiments may include an orientation sensor secured to theelongate barrel housing and configured to determine changes in anorientation of the elongate barrel.

Some embodiments may include a method of collecting data with a smartwand held in a user's hand. The method may include receiving sensor datameasuring a gesture made with the smart wand in midair by the user'shand. The gesture may be made without contact between the smart wand anda surface other than the user's hand. The method may also includetransmitting the sensor data to a remote computing device for generatinga two-dimensional rendering corresponding to the gesture made with thesmart wand in midair by the user's hand. The two-dimensional renderingmay include at least one drawn line or mark that matches a pattern ofthe gesture. The gesture may include shorthand used for the conversioninto the two-dimensional rendering. The two-dimensional rendering mayinclude at least one alphanumeric character. The conversion into thetwo-dimensional rendering may use a previous training sequence thatcorrelates the gesture to the two-dimensional rendering.

In various embodiments, the method may include converting the sensordata to rendering data configured to generate a visual representation ofthe gesture made with the smart wand on a display of a remote computingdevice. The sensor data being transmitted to the remote computing devicemay include the rendering data from the conversion. The method mayinclude receiving an authorization input corresponding to the usercontacting a first button on the smart wand. Also an unlock activationmessage may be transmitted to the remote computing device in response toreceiving the authorization input. The unlock activation message mayinclude the authorization input for unlocking the computing device froma locked state. Reception by the remote computing device of the unlockactivation message may not require line-of-sight between the smart wandand the remote computing device.

In various embodiments, the first button may include a fingerprintsensor and the authentication input may include fingerprint datacollected by the fingerprint sensor on the smart wand. Theauthentication input may include a sequence of button presses with aparticular timing sequence that acts as an authentication code forunlocking the computing device from the locked state. The method mayinclude receiving an application launch input corresponding to the usercontacting an application launch button on the smart wand. Also, anapplication activation message may be transmitted to the remotecomputing device in response to receiving the application launch input.The application activation message may act to activate an application onthe remote computing device. The application activated by theapplication activation message may receive an audio input from amicrophone of the remote computing device. The application activated bythe application activation message may receive a video input from acamera of the remote computing device.

BRIEF DESCRIPTION OF THE DRAWINGS

The accompanying drawings, which are incorporated herein and constitutepart of this specification, illustrate example embodiments of theinvention, and together with the general description given above and thedetailed description given below, serve to explain the features of theinvention.

FIG. 1A is a schematic representation of a user using a smart wanddevice linked to a remote computing device according to variousembodiments.

FIG. 1B is a component block diagram illustrating a smart wand deviceaccording to various embodiments.

FIG. 2 is a component block diagram illustrating a smart wand deviceaccording to various embodiments.

FIG. 3A is a schematic representation of a coordinate system used formeasuring a gesture made with a smart wand device according to variousembodiments.

FIG. 3B is a schematic representation of the coordinate system of FIG.3A used for measuring further elements of a gesture made with a smartwand device according to various embodiments.

FIG. 4 is a two-dimensional rendering of gestures made with a smart wanddevice according to various embodiments.

FIG. 5 is a three-dimensional rendering of gestures made with a smartwand device according to various embodiments.

FIG. 6 is a schematic representation of a remote computing devicegenerating a two-dimensional rendering of a gesture and converting thetwo-dimensional rendering into recognized text according to variousembodiments.

FIG. 7 is a schematic representation of a remote computing devicegenerating a two-dimensional rendering of a shorthand gesture accordingto various embodiments.

FIG. 8 is a schematic representation of a gesture analysis moduleaccording to various embodiments.

FIG. 9 is a schematic representation of an inference engine for enhancedinput recognition according to various embodiments

FIG. 10 is a process flow diagram illustrating a method of generating atwo-dimensional rendering using a smart wand device according to variousembodiments.

FIG. 11 is a process flow diagram illustrating a method of gesturerecognition according to various embodiments.

FIG. 12 is a component diagram of an example remote computing deviceaccording to various embodiments.

DETAILED DESCRIPTION

Various embodiments will be described in detail with reference to theaccompanying drawings. Wherever possible the same reference numbers willbe used throughout the drawings to refer to the same or like parts.References made to particular examples and implementations are forillustrative purposes, and are not intended to limit the scope of theinvention or the claims.

FIG. 1A illustrates a user 5 using a smart wand device 100 in accordancewith various embodiments. The smart wand device 100 works in conjunctionwith a remote computing device 200, via a direct wireless link 15, toactivate, unlock, enter notations into, and/or interact with the remotecomputing device 200 without the need for a special writing surface.Gestures made by the user 5 with the smart wand device 100 may beconverted into two-dimensional renderings that may be displayed and usedby the remote computing device 200. The two-dimensional renderings maycorrespond to written words, drawings, or other notations, like thosetypically done using a pen, pencil, or other writing instrument on awriting surface. The smart wand device 100 detects gestures usingonboard sensors that measure movement and/or orientation of the smartwand device 100 in midair. Using the direct wireless link 15, the smartwand device 100 may communicate the measured movement to the remotecomputing device 200. The remote computing device 200, which may be asmart phone or other computing device, may display and save the measuredmovements as two-dimensional renderings, corresponding to text,drawings, notations, or the like. In some embodiments, one or more ofthe two-dimensional renderings may be information structures that storeinformation suitable for storing, analyzing, processing, and/orrendering the measured movements. In some embodiments, the smart wanddevice 100 may be configured so as to allow the user 5 to activate orunlock the remote computing device 200 by transmitting a pass code,personal identification number (PIN), a username and passwordcombination, a biometric such as a fingerprint scan, and/or other accesskeys via the direct wireless link 15.

FIG. 1B illustrates a functional block diagram of an example smart wanddevice 100 suitable for implementing various embodiments. With referenceto FIGS. 1A and 1B, the smart wand device 100 may be sized and shaped tolook and feel like a traditional writing instrument or a wand. In theexample illustrated in FIG. 1B, the smart wand device 100 is shapedsimilar to a wand. The smart wand device 100 may include a nib 110disposed at the tip of one end of an elongate barrel housing 120, and aplurality user interface elements 101, such as input buttons, keys, etc.The smart wand device 100 may also include a biometric sensor 102, sucha finger print reader configured to collect a fingerprint scan from theuser. The smart wand device 100 may further include various hardwareand/or software components suitable for collecting and processing sensordata from sensors, including speakers, sensors for monitoring physicalconditions (e.g., location, direction, motion, orientation, vibration,pressure, etc.), cameras, compasses, GPS receivers, communicationscircuitry (e.g., Bluetooth®, WLAN, WiFi, etc.), and other well-knowncomponents (e.g., accelerometer, gyroscope, etc.) of modern electronicdevices.

In the example illustrated in FIG. 2, the smart wand device 100 isshaped similar to conventional ink-based pen. The smart wand device 100may include a nib 110, disposed at the tip of one end of an elongatebarrel housing 120. The smart wand device 100 does not necessarily havean ink or lead-based dispensing tip and thus the nib 110 may identify toa user 5, which end of the smart wand device 100 is used as a virtualwriting tip. In addition, the smart wand device 100 may optionallyinclude a pen-style clip 125 toward the other end, opposite the nib 110.In addition to the functional benefits provided by traditional pen-styleclips (e.g., used for securing the smart wand device 100 to a pocket orsheet of material), the pen-style clip 125 may provide a visual clue tothe user 5 as to which end is not used as the virtual writing tip. Inaddition, the pen-style clip 125 may optionally function as a user inputbutton. An outer surface of the elongate barrel housing 120 may beshaped or formed of materials that provide ergonomic enhancements, likea comfortable or secure grip, or stylistic elements.

The nib 110 of the smart wand device 100 may be retractable into arecess within the elongate barrel housing 120. Retracting the nib 110may protect delicate or sensitive components or materials forming thenib 110. For example, the nib 110 may include one or more sensors (e.g.,a position or force sensor). In addition, the nib 110 may include acapacitive touch material for using the smart wand device 100 as astylus. Retracting the nib 110 may generate an off signal, registered bythe processor 150, and corresponding to the smart wand device 100 beingin a non-rendering mode or a power-saving state. In contrast, deployingthe nib 110 may generate an on signal, registered by the processor 150,and corresponding to the smart wand device 100 being in an activerendering mode. The user may actuate a retraction mechanism 115 using aretraction button 190 at the other end of the smart wand device 100.Optionally, a force sensor may be incorporated into the nib 410 formeasuring a force exerted thereon, which may allow the nib 410 tooperate as another input button.

In accordance with various embodiments, the smart wand device 100 mayhave one or more position sensors configured to measure indicators of anactual and/or relative position of the smart wand device 100. Theposition sensor(s) may measure movements made by the user 5 with thesmart wand device 100 in midair, without contact between the smart wanddevice 100 and a surface other than the user's hand (i.e., the handholding the smart wand device 100). The movements measured by theposition sensor(s) represent one or more gestures made by the user 5with the smart wand device 100.

The position sensors may include an optical sensor 131 with a lens forimaging the three-dimensional environment and a photo-detector forsensing light passing through the lens. For example, the photo-detectormay be a pixel array sensor, e.g., a CMOS, a CCD or a PIN-diode sensor.Optionally, the optical sensor 131 may work in conjunction with a lightprojector 132 that emits a certain light into the real three-dimensionalenvironment. The optical sensor 131 may receive a scattered portion ofthe light from light projector 132 to infer an absolute position inthree-dimensional space of the smart wand device 100. The optical sensor131 may determine a centroid of light flux from the light projector 132.In addition, to avoid ambient illumination interference, the lightprojector 132 may use a number of infrared emitting diodes or infraredlight-emitting diodes (LEDs). The infrared LEDs may be modulated in acertain pattern to further improve their signal-to-noise performance andrender them better distinguishable. The optical sensor 131 may includemore than one such sensor arranged in various locations along the lengthof the elongate barrel housing 120 to ensure orientations and positionsof the smart wand device 100 may be determined. For example, the opticalsensor 131 may be disposed near the nib 110 in order to detect light inthe same direction the nib 110 faces. Similarly, the light projector 132may have a complementary position to the optical sensor 131, such as anopposite side laterally of the smart wand device 100.

In addition, the smart wand device 100 may include a motion sensor 134,such as a gyroscope or an accelerometer. The motion sensor 134 mayinclude an optical flow-measuring unit. Alternatively or additionally,the motion sensor 134 may include magnetic field measuring units oracoustic field measuring units that may interpolate or refine anabsolute or inferred position measured by the optical sensor 131. Duringnormal operation (i.e., an active rendering mode), the smart wand device100 may sample the encoding from the motion sensor 134 as the nib 110moves in the air. The sampled encoding may be decoded by the processor150 to identity the absolute position of the nib 110 and the smart wanddevice 100 generally.

In accordance with various embodiments, the smart wand device 100 mayinclude one or more user interface input mechanisms, such as anactivation button 140 and a function-select button 142. Alternatively,once a user turns on or activates the smart wand device 100, applyingpressure to one or more discrete elements, like the nib 110 or thepen-style clip 125, may activate functions or operate as functionselector or input button.

The activation button 140 may be disposed at an intermediate positionalong the elongate barrel housing 120. Preferably, the activation button140 is ergonomically positioned along the elongate barrel housing 120,such that while the user is holding the smart wand device 100 with aselect hand, a finger on that hand is comfortably and convenientlyreceived on the activation button 140. In this way, while holding thesmart wand device 100, the user may selectively engage and disengage theactivation button without letting go of the smart wand device 100 orhaving to use the other hand.

In various embodiments, the activation button 140 is a fingerprintscanner configured to capture fingerprint features, when a user's fingeris placed on or over a surface of the activation button 140. Thus, partof the ergonomic positioning of the activation button 140 may includeconfiguring the activation button at an angle and position thatconsistently captures the user's fingerprint while the user holdsnormally the smart wand device 100. The activation button 140 may employone or more of various technologies, including optical, capacitive, RF,thermal, piezoresistive, ultrasonic, and/or piezoelectric. For example,Touch ID technologies used on iPhones, such as a capacitive touch sensorconfigured to detect the user's fingerprint or features thereof, may beincluded in the activation button 140.

The captured fingerprint features may be digitally processed by theprocessor 150 of the smart wand device 100 to create a biometrictemplate (a collection of extracted features) which may be stored inmemory and used for matching. In this way, the biometric template may becompared to a current user's presently scanned fingerprint features todetermine whether it is a match. Without a proper match, the currentuser may be prohibited from using all or certain functions of the smartwand device 100.

Alternatively, such as for security reasons, the captured fingerprintfeatures may be stored onboard the smart wand device only long enough tocommunicate the features to a remote computing device. The remotecomputing device may have stronger security and/or the user may feelmore comfortable with such personal and important biometric data beingstored in only one place, like the user's smart phone. Thus, the remotecomputing device may create or already have stored the biometrictemplate, which may be used for later matching. In this way, thecurrently captured fingerprint features may be promptly transmitted tothe remote computing device and wiped from an onboard memory, as asecurity feature. Thereafter, the remote computing device may determinewhether the data it received, representing the fingerprint features,matches a stored biometric template used as an access key for the remotecomputing device and/or a particular program or application thereon. Inresponse to a processor of the remote computing device determining thatthe data it received, representing the fingerprint features, matches thestored biometric template, that processor may unlock the remotecomputing device and/or transmit an indication to the smart wand devicethat the data representing the fingerprint features were accepted forunlocking the remote computing device.

The smart wand device 100 may be powered by a rechargeable battery 180,which may be charged from a dedicated pen cradle or from a USB chargerconfigured to be plugged into a charging port 182. The smart wand device100 may additionally include a display 160 and/or one or more LEDindicators 162.

The smart wand device 100 may include one or more processor(s) 150coupled to the other electronic components of the smart wand device 100,such as the optical sensor 131, optional light projector 132, motionsensor 134, activation button 140, function-select button 142, display160, LED indicator(s) 162, rechargeable battery 180, and the chargingport 182 if configured to communicate data (e.g., a USB port). Theprocessor(s) 150 may be configured with processor-executableinstructions to receive inputs from the position sensor(s), as well asgenerate outputs for the display 160, the LED indicator(s) 162, or otheroutput elements. The position sensors, such as the optical sensor 131and/or the motion sensor 134 may be used as means for receiving signalsand/or indications of a position and/or movement. The processor(s) maybe used as means for performing functions or determiningconditions/triggers, such as whether patterns match or as means foridentifying gestures, a reference input or determining a frame ofreference. In addition, the display 160 or a speaker may be used asmeans for outputting. The processor may be coupled to one or moreinternal memories 152. Internal memories 152 may be volatile ornon-volatile memories, which may be secure and/or encrypted memories, orunsecure and/or unencrypted memories, or any combination thereof. Theprocessor 150 may be any programmable microprocessor, microcomputer ormultiple processor chip or chips that can be configured by softwareinstructions (i.e., applications) to perform a variety of functions,including the functions of various aspects described above. Multipleprocessors may be provided, such as one processor dedicated to one ormore functions and another one or more processors dedicated to runningother applications/functions. Typically, software applications may bestored in the internal memory 152 before they are accessed and loadedinto the processor. The processor 150 may include internal memory 152sufficient to store the application software instructions. In manydevices, the internal memory 152 may be a volatile or nonvolatilememory, such as flash memory, or a mixture of both. For the purposes ofthis description, a general reference to memory refers to memoryaccessible by the processor including internal memory or removablememory plugged into the hearing aid and memory within the processor.

The processor 150 may be coupled to a transceiver 170, within theelongate barrel housing 120, for communicating with the remote computingdevice (e.g., 200). The transceiver 170 may use existing conductiveelements, such as the pen-style clip 125, as an antenna or an extensionof antenna resources of the smart wand device 100. The transceiver 170may service as a dedicated RF resource chain for communications usingtechnologies, such as Wi-Fi or Bluetooth. The transceiver 170 mayinclude more than one transceiver for supporting separate technologies,such as one for Bluetooth and one for Wi-Fi. Alternatively, additionalcommunication resources and associated transceivers may be provided andused for remote connections to remote computing device.

A processor 150 of the smart wand device 100 may initially receive apair request entered by a user through an interface of the smart wanddevice 100. The pair request may cause the processor 150 to initiate apairing routine with a nearby remote computing device. For example, theprocessor 150 may use the transceiver 170 to convey an external pairingsignal to the remote computing device to establish a wirelesscommunication link. In some embodiments, the pairing routine mayalternatively or additionally establish a direct wireless link using aWi-Fi data connection.

The smart wand device 100 may have various modes of operation, includingan off mode, low-power mode, training mode, and an active renderingmode. Like many conventional electronic devices, the off mode mayconserve maximum power and be used to reset the processor. A low-powermode may be provided to conserve as much power as possible withoutre-booting the processor (e.g., 150). For example, the low-power modemay deactivate sensors (e.g., 132, 134), the display (e.g., 160), and/orthe transceiver (e.g., 170). The training mode may enable the processorto calibrate, learn, and translate user gestures into meaningful text orcommands. The active rendering mode may correspond to when the sensorsare actively measuring changes in position (i.e., gestures made with thesmart wand device 100) and/or communicating the measured changes inposition to a remote computing device.

In accordance with various embodiments, a user may be provided with oneor more ways to make the smart wand device 100 enter, exit, or remain inthe active rendering mode. For example, active engagement by the userwith the activation button 140 may make the smart wand device 100 enterthe active rendering mode, while disengagement therewith may make thesmart wand device 100 exit the active rendering mode. Thus, continualengagement by the user with the activation button 140 maintains thesmart wand device in the active rendering mode. Alternatively oradditionally, a double-tap on the activation button 140 by the user maybe used to either enter or exit the active rendering mode, without theneed to continue holding the activation button 140 to maintain theactive rendering mode. Also, an orientation of the smart wand device100, detected by onboard sensors, may control whether the smart wanddevice 100 is in the active rendering mode. For example, as long as theuser holds the smart wand device 100 horizontally, plus or minusforty-five degrees, a processor of the smart wand device may maintainthe active rendering mode. In this way, once the processor determinesthat sensors have detected a tilting of the smart wand device 100 up ordown beyond forty-five degrees, the processor may automatically exit theactive rendering mode.

As part of the active rendering mode, the processor may use thetransceiver to transmit sensor data, which measures and reflects themovements of a gesture, to the remote computing device via the directwireless link for further processing. In this way, the remote computingdevice may translate the sensor data to text or function commands. Inthis way, a processor of the remote computing device or a web-basedapplication may identify input characters, (e.g., alpha-numericcharacters or symbols) based on the measured changes in position.Alternatively, during the active rendering mode, the processor of thesmart wand device 100 may translate measured changes in position to textor function commands. In this way, the processor of the smart wanddevice 100 may identify input characters, (e.g., alpha-numericcharacters or symbols) based on the measured changes in position. Whenthe smart wand device 100 is paired and actively communicating with theremote computing device 200, the smart wand device 100 may transmit datain real time or as soon as all previously buffered data has beentransmitted. In addition or alternatively, the processor of the smartwand device 100 may temporarily store the sensor data, such as when thedirect wireless link is unavailable, until a later time when the directwireless link becomes available.

In various embodiments, the smart wand device 100 may additionallyinclude components such as a microphone and/or a camera for receivingand capturing (i.e., recording) audio and/or video inputs. Themicrophone and/or camera may be used to record memos, messages, and/oreven to interact with a remote computing device. A user may interactwith and/or activate an application on the remote computing device usingan audio and/or video input received by the microphone and/or camera andtransmitted to the remote computing device.

In various embodiment, the smart wand device may include one or moreadditional components, such as a speaker, scanner, laser pointer,thermometer, biometric sensors (e.g., for measuring pulse, bloodpressure, etc.), and/or the like.

FIGS. 3A and 3B illustrate the smart wand device 100 oriented in twodifferent positions within a three-dimensional (3D) space, having aCartesian coordinate system with x, y, and z-axes. When the smart wanddevice is in active rendering mode, the position sensor(s) (e.g.,optical sensor 131 and/or motion sensor 134) may measure various aspectsof the movement of the smart wand device 100 to determine a relativeposition. One aspect of the movement relates to relative changes inposition of a reference point of the smart wand device 100. Thosechanges of position may correspond to one or more gestures made by theuser 5 with the smart wand device 100 in midair. For ease of explanationand illustration, the nib (e.g., 110 in FIG. 2) of the smart wand device100 may be used as the reference point for measuring the changes ofposition. Alternatively, other positions on or near the smart wanddevice 100 may be used as the reference point.

The smart wand device 100 may establish a reference position, from whichmovements and other positions are measured. For example, when the userinitially places the smart wand device 100 in the active rendering mode(e.g., by pressing the activation button 140), an origin point (x₀, y₀,z₀) may be established. In this way, the position sensors measurechanges of position relative to that origin point. In FIG. 3A, the nibof the smart wand device has been moved from the origin point (x₀, y₀,z₀) to a first reference point (x₁, y₁, z₁). The movement between theorigin point (x₀, y₀, z₀) and the first reference point (x₁, y₁, z₁)define a first line 301, which may form all or part of a gesture. Theprocessor may differentiate the first reference point (x₁, y₁, z₁) fromother points along the first line 301 by a hesitation or pause inmovement that has taken place at the first reference point (x₁, y₁, z₁).Alternatively, the processor may differentiate the first reference point(x₁, y₁, z₁) from prior reference points along the first line 301 orsubsequent positions beyond the first line 301 by a smooth and/orcontinuous movement (i.e., no hesitation or sharp changes of direction).In FIG. 3B, the nib of the smart wand device has been moved from thefirst reference point (x₁, y₁, z₁) to a second reference point (x₂, y₂,z₂). The movement between the first reference point (x₁, y₁, z₁) and thesecond reference point (x₂, y₂, z₂) may define a second line 302, whichtogether with the first line 301 may form all or part of a gesture.

For ease of explanation, multiple points (i.e., the origin, firstreference point, and second reference point) within the 3D space inFIGS. 3A-3B are each denoted by a dot, larger than the correspondinglines between points. However, when converting the movement path to a 2Drendering in accordance with various embodiments, the points do not needto be larger (i.e., do not need to stand out) from the movement pathlines. In addition, although both the first line 301 and the second line302 are illustrated as straight lines, one or both of the first andsecond lines 301, 302 may include one or more curved portions or evenhave no straight portions.

In accordance with various embodiments, the user may designate thebeginning and the end of a particular gesture by pressing and releasingthe activation button (e.g., activation button 140 in FIG. 2). Forexample, while intending to make a gesture with the smart wand device,the user may press and hold the activation button with a thumb, similarto the way individuals hold their thumb against a traditional writinginstrument when writing. The user may denote an end to one gesture bylifting the thumb from the activation button, while still holding otherportions of the smart wand device with other fingers and portions of thesame hand. In this way, the user may press and hold the activationbutton with the thumb while moving the smart wand device through the airto follow a pattern representing a particular character or word. Oncethe pattern is complete, the user may release the activation button todenote as much. The user may then continue with another character orword by once again pressing and holding the activation button whilemaking further movements associated with the same or other pattern.

FIG. 4 illustrates an example of a 2D rendering of multiple gesturesmeasured during intervals in which the smart wand device is in theactive rendering mode, separated by intervals in which the smart wanddevice is not in the active rendering mode. For example, continuedpressing of the activation button may maintain the active renderingmode, while intervals in which the user released the activation buttonmay exit the active rendering mode. Alternatively, continuallymaintaining the smart wand device in a horizontal configuration, plus orminus forty-five degrees, may maintain the active rendering mode, whileintervals in which the user tilts the smart wand device more thanforty-five degrees from horizontal may exit the active rendering mode.The reference points in FIG. 4 are emphasized as dots larger than theconnection lines, similar to FIGS. 3A-3B, for ease of explanation.

The 2D rendering illustrated in FIG. 4 may represent a conversion ofuser movements in which the user may have starting pressing and holdingthe activation button when the nib of the smart wand device was atposition A. Thereafter, while continuing to hold the activation button,the user moved the nib of the smart wand device to position B, followedby positions C, D, and E, in that order. Once arriving at position E,the user let go of the activation button then moved to position F, wherethe user briefly pressed the activation button without further movement.After letting go of the activation button while still at position F, theuser moved the nib to position G and re-engaged the activation button.From position G, the user continued to hold the activation button andmoved in a serpentine pattern to position H, where the user once againlet go of the activation button. Finally, after arriving at position H,the user let go of the activation button then moved to position I, wherethe user briefly pressed the activation button without further movement,then let go. In this way, the user will have made four separate gesturesthat form the letters “A” and “S,” each followed by a period, as shownin FIG. 4. Alternatively, the above step of pressing and holding theactivation button, for entering and maintaining the active renderingmode, may be substituted with maintaining the smart wand device in ahorizontal orientation, plus or minus forty-five degrees. Similarly, theabove step of letting go of the activation button, for exiting theactive rendering mode, may be substituted with tilting the smart wanddevice more than forty-five degree from the horizontal orientation.

Although the user may intend to form a 2D rendering, the user's hand maymove the smart wand device (e.g., 100) unintentionally in more than twodimensions. This may occur because without a conventional writingsurface, it may be difficult for a user to limit movements of the smartwand device to only two dimensions. The user's hand attempting to formcharacters in midair is not constrained to follow the plane of aconventional 2D writing surface. Thus, in accordance with variousembodiments, one dimension of movement may be ignored or useddifferently than the other two dimensions of movement, which other twodimensions may be used for conversion into the 2D rendering.

FIG. 5 illustrates an example of a 3D movement path that may beconverted into a 2D rendering similar to that of FIG. 4, by ignoring thez-axis dimension of movement. The reference points in FIG. 5 aredesignated by coordinates along each axis (i.e., the x-axis, the y-axis,and the z-axis), exemplifying a corresponding position in 3D space.Thus, the user may have starting pressing and holding the activationbutton when the nib of the smart wand device was at first position,corresponding to an origin with coordinates (0, 0, 0). Thereafter, whilecontinuing to hold the activation button, the user moved the nib of thesmart wand device to a second position (2, 6, 6), followed by a thirdposition (4, 0, 6), fourth position (5, 0, 6), and fifth position (1, 3,3), in that order. Once arriving at the fifth position (1, 3, 3), theuser let go of the activation button then moved to a sixth position (6,0, 6), where the user briefly pressed the activation button withoutfurther movement. After letting go of the activation button while stillat the sixth position (6, 0, 6), the user moved the nib to a seventhposition (8, 1, 6) and re-engaged the activation button. From theseventh position (8, 1, 6), the user continued to hold the activationbutton and moved in a 3D serpentine pattern to a ninth position (12, 5,0), where the user once again let go of the activation button. Finally,after arriving at the ninth position (12, 5, 0), the user let go of theactivation button then moved to a tenth position (13, 6, 0), where theuser briefly pressed the activation button without further movement,then let go. In this way, by ignoring the z-axis component of themovement, the user will have made four separate gestures that form theletters “A” and “S,” each followed by a period, similar to that shown inFIG. 4.

The processor of the smart wand device and/or the remote computingdevice may determine which dimension (i.e., axis) not to use for 2Drendering based on an orientation of the smart wand device. For example,if the smart wand device is being held such that a longitudinal axis(i.e., lengthwise) extends horizontally or substantially parallel to theground, the processor may not use movements along a measured axis thatis parallel to the ground, in a direction of that longitudinal axis ofthe smart wand device. Similarly, if the user is holding the smart wanddevice such that the longitudinal axis extends vertically orsubstantially perpendicular to the ground, the processor may not usemovements along a measured axis that corresponds to the vertical axis.For angles in between the horizontally and vertically extendinglongitudinal axis, the processor may use a forty-five degree angle therebetween to decide which axis to not use for 2D rendering.

FIG. 6 illustrates a remote computing device 200 generating atwo-dimensional rendering 210 of a gesture 50 and converting the 2Drendering 210 into machine-encoded text 215 (i.e., recognized text), inaccordance with various embodiments. Sensor data collected by the smartwand device 100 may be transmitted to the remote computing device 200for interpretation. Once received by the remote computing device 200,the sensor data may be subsequently converted into a 2D rendering 210.In FIG. 6, dotted lines represent the movements forming the gesture 50,made in midair by the user 5 with the tip of the smart wand device 100.As shown, a path of the gesture 50 is similar to cursive writing formingletters and the words, “call home.” However, gestures made with thesmart wand device need not be limited to creating letters, words,sentences, etc. For example, the gestures may include drawn figures,shapes, diagrams, or almost anything a user can draw with the smart wanddevice 100. The two-dimensional rendering 210 of a display 205 on theremote computing device 200 may have a close resemblance to the gesture50, in order to ensure it is an accurate representation of the gesture50. In accordance with various embodiments, the processor of the remotecomputing device 200 may be configured with processor-executableinstructions to perform character recognition on the 2D rendering 210.Character recognition is the electronic conversion of 2D renderings intomachine-encoded text 215. After performing character recognition, theprocessor of the remote computing device 200 may store, display and/orotherwise use the machine-encoded text 215.

FIG. 7 illustrates the remote computing device 200 converting ashorthand gesture 51 into a translated 2D rendering 220 of text thatdoes not precisely resemble the shorthand gesture 51. Like the characterrecognition, described above with regard to FIG. 6, a processor of thesmart wand device 100 and/or the remote computing device 200 mayrecognize certain gestures that translate to particular machine-encodedtext and/or words. In addition, certain recognized gestures maytranslate to one or more operational commands for the remote computingdevice 200.

In accordance with various embodiments, the smart wand device may have atraining mode. The training mode may help the user get used to makinggestures with the smart wand device that may be recognized by aprocessor, such as in the smart wand device and/or the remote computingdevice. In addition, the training mode may allow the user to haveparticular gestures recorded in a memory and associated with, text,words, or operational commands. For example, while in training mode, auser may execute movements with the smart wand device (e.g., theshorthand gesture 51 in FIG. 7), which are stored in a memory.Thereafter, the user may use the remote computing device, a userinterface, or other means of data entry to associate the particulartext, words, or operational commands with the stored shorthand gesture.

FIG. 8 illustrates functional modules for gesture recognition in agesture analysis module 800, which may be included in the smart wanddevice 100 and/or the remote computing device (e.g., 200). An outputfrom the one or more sensors (e.g., optical sensor 131 and/or motionsensor 134 in FIG. 2) may be an analog output. An analog/digitalconversion module 810 may digitize such an analog output. The digitizedsignal may be broken down and analyzed by a gesture extraction module820 in order to identify patterns or features of measured movements.Such patterns or features may be input to a gesture classifier module840. In this way, certain gesture classifications may activate functionsof the smart wand device 100 and/or the remote computing device, whileother patterns or features need not be acted upon if considered noise.Additionally, a training module 830 may receive the identified patternsor features from the gesture extraction module 820. The training module830 may include a labeling module 832 and a classification functionsmodule 838 for informing the gesture classifier module 840 about thedistinct gestures that need to be acted upon. The training module 830may use a machine-learning algorithm, such as a K-means clustering or asupervised learning model, such as support vector machines (SVM). Forexample, using support vector machines the feature space may be the peakor average amplitude of the signals received from each sensor. An outputof the training module 830 may be a set of decision functions that maybe used to classify either real or test data into distinct gestures. Aprocessor of the smart wand device 100 and/or the remote computingdevice may receive only appropriate classified gestures that mayactivate functions.

The processor of the smart wand device 100 and/or the remote computingdevice may provide more robust gesture recognition by including inputfrom more than one gesture sensor in either real or test data. Inaddition, the processor may categorize the input from gesture sensorsand use a gesture analysis module and/or inference engine to recognizegestures corresponding to particular movements by the user. Supervisedmachine learning algorithms may be employed for distinguishing thedifferent movements from the potentially noisy signal data.

FIG. 9 illustrates functional modules for enhanced input recognition ina smart wand device 100 and/or the remote computing device using aninference engine 900. The smart wand device 100 may capture images, suchas from a nearby object, wall, or other surface. An inference engine 900may analyze an output from the onboard orientation and/or positionsensor. As part of the inference engine 900, an image analysis module910 may receive and analyze the raw data and detect surface proximity.Also, the image analysis module 910 may detect objects, such as a chair,table, desk, etc., in the captured image. Based on the image analysis, aframe of reference may be determined and/or verified by a calibrationmodule 920. With a frame of reference determined and/or verified, aposition/motion determination module 930 may locate the smart wanddevice 100 relative to the frame of reference and/or track its movement.The location or movement of the smart wand device 100 may be translated,based on corresponding input values, to a 2D rendering or functionalcommand by an output module 940, which may be acted upon by a processorof the smart wand device 100.

Additionally, redundant orientation and/or position sensors may be usedin conjunction with a primary orientation and/or position sensor tocalibrate and/or confirm position determinations made from capturedimages. In this way, the gesture analysis module 800 receiving inputfrom the gesture sensor(s) may contribute its own output to thecalibration module 920. For example, orientation and/or position sensorsmay detect a particular tilt of the smart wand device 100 relative to anearby detected surface. Thus an algorithm, such as a Bayesian inferencealgorithm, may provide soft estimates of the altitude and/or azimuthangles created. Those soft estimates may then be compared in thecalibration module 920 to determinations made from the image analysis.Alternatively, in response to the orientation and/or position sensorbeing turned off or in a stand-by mode, the gesture analysis module 800may provide the output module 940 with an indication that a commandshould be output to turn on the orientation and/or position sensor(s).

FIG. 10 illustrates an embodiment method 1000 of generating atwo-dimensional rendering using a smart wand device, according tovarious embodiments. In determination block 1010, the processor maydetermine whether an activation input is received to activate (i.e.,enter) the active rendering mode. Activation of the active renderingmode may automatically turn on the gesture sensors of the smart wanddevice. The activation input may be received automatically when thesmart wand device is turned on. Alternatively, the gesture sensor(s) mayremain off or dormant while the smart wand device is on and onlyactivated by an activation input received from a control operated by theuser (e.g., maintaining a finger or pressure on the activation button ormaintaining the smart wand device in a particular orientation, such ashorizontal) or some other trigger.

Considering the gesture sensor(s) may expend a significant amount ofpower, it may be desirable to provide one or more different ways ofavoiding unintentionally enabling the orientation and/or position sensorand/or the active rendering mode functions. For example, redundantactivation inputs or at least two different activation inputs may berequired before enabling the active rendering mode.

In response to determining that an activation input is received (i.e.,determination block 1010=“Yes”), the active rendering mode may beactivated in block 1020. In conjunction with the activation of theactive rendering mode, it may be useful to provide a visual, audioand/or haptic (e.g., vibration) indication to the user that the activerendering mode has been and/or is activated. In response to determiningthat no activation input is received (i.e., determination block1010=“No”), the processor may await such an activation input beforeinitiating the rest of the method 1000 or repeat the determination indetermination block 1010.

With the active rendering mode active, sensor data may be received inblock 1030. The received sensor data may represent one or more gesturesmeasured by the gesture sensors, analyzed in series or in parallel by aprocessor of the smart wand device. Alternatively, the received sensordata may include more than one set of sensor data analyzed collectivelyin accordance with the subsequent blocks described below.

In determination block 1040, the processor may determine whether aposition of a reference point (e.g., the tip) of the smart wand deviceis detected in the sensor data. In response to determining that noposition is detected in the sensor data (i.e., determination block1040=“No”), the processor may determine whether to exit the activerendering mode in determination block 1045. In response to detecting aposition of the reference point of the smart wand device in the sensordata (i.e., determination block 1040=“Yes”), the processor may calibrateitself by orienting itself and determining a frame of reference. Thus,in response to determining that a position of the reference point of thesmart wand device is detected in the sensor data (i.e., determinationblock 1040=“Yes”), the processor may determine whether an orientation isdetected in the sensor data or whether a reference orientation waspreviously established in determination block 1050. In response todetermining that no orientation is detected in the sensor data and thatno reference orientation was previously established (i.e., determinationblock 1050=“No”), the processor may determine whether it is appropriateto exit the active rendering mode in determination block 1045.

The determination in determination block 1045 regarding whether to exitthe active rendering mode may be based on an input received from theuser, a particular software event, a timed trigger for conserving powerin response to certain conditions (i.e., no activity, position, ororientation detected for a predetermined period of time) or othersettings of the smart wand device. In response to determining that theactive rendering mode should be exited (i.e., determination block1045=“Yes”), the processor may again determine whether an activationinput is received in determination block 1010. In response todetermining that the smart wand device should remain in the activerendering mode (i.e., determination block 1045=“No”), the processor mayreceive further sensor data from the gesture sensor(s) in block 1030.

In response to detecting an orientation in the sensor data or that areference input was previously established (i.e., determination block1050=“Yes”), the processor may determine a frame of reference in block1060. Also, the processor may determine a position of the referencepoint of the smart wand device detected in the sensor data with respectto the determined frame of reference in block 1070. In block 1080, a 2Drendering value associated with the determined position may bedetermined. Thus, a visual indication regarding the determined 2Drendering value may be provided on a display of a remote computingdevice in block 1090 by applying the determinations from blocks 1060,1070, 1080.

FIG. 11 illustrates an embodiment method 1100 of gesture recognitionthat may be performed by a processor of the smart wand device and/or theremote computing device. In block 1110, sensor data may be received fromone or more gesture sensors (e.g., an orientation sensor and/or aposition sensor) directly or via the direct wireless link. The receivedsensor data may thus be processed in block 1120 to extract features. Indetermination block 1130, the processor may determine whether at leastone gesture is recognized based on the extracted features.

In response to determining that no gesture is recognized from theextracted features (i.e., determination block 1130=“No”), the processormay determine whether any frame of reference data may be derived fromthe extracted features in determination block 1140. In response todetermining that no frame of reference data may be derived from theextracted features (i.e., determination block 1140=“No”), the processormay await receipt of further input from the gesture sensor(s) in block1110. In response to determining that frame of reference data may bederived from the extracted features (i.e., determination block1140=“Yes”), the processor may output such frame of reference data inblock 1150. The output of such frame of reference data may includestoring that data in a memory for use in future feature extractions(i.e., block 1120) and/or gesture recognition determinations (i.e.,determination block 1130). When frame of reference data is output inblock 1150, the processor may await receipt of further input from thegesture sensor in block 1110.

In response to determining that an extracted feature matches arecognized gesture (i.e., determination block 1130=“Yes”), a commandassociated with the recognized gesture may be output in block 1160. Forexample, the recognized gesture may activate certain features of thesmart wand device, the remote computing device, and/or trigger aparticular 2D rendering in a display of a paired remote computingdevice. In particular, the recognized gesture may indicate the gesturesensor(s) should be activated. In which case, the sensor data receivedin block 1110 may be considered an activation input as described abovewith regard to determination block 1010 in FIG. 10. Additionally, inresponse to the recognized gesture indicating the gesture sensor(s)should be activated, the command output in block 1160 may be an activerendering mode activation command. When a command associated with therecognized gesture is output in block 1160, the processor may awaitreceipt of further sensor data from the gesture sensor(s) in block 1110.

The various embodiments may include a smart wand device that includes anelongate barrel housing extending between a tip and an end cap, in whicha portion of the elongate barrel housing is sized to be held by a user.The smart wand device may also include a position sensor secured to theelongate barrel housing and configured to determine changes in positionof a reference point of the smart wand device, a transceiver, and aprocessor. The processor may be disposed within the elongate barrelhousing and coupled to at least the position sensor and the transceiver.In some embodiments, the processor may be configured withprocessor-executable instructions to perform operations that includereceiving sensor data from the position sensor and transmitting thesensor data to a remote computing device (e.g., to a mobile device vie adirect wireless link). The received sensor data may include measurementsof movements or gestures made by the user, with the tip in midair, whilethe user holds the elongate barrel housing. The sensor data may identifya gesture made without contact between the tip and a surface. Thetransmitted sensor data may include information suitable for cause thereceiving device (e.g., remote computing device) to perform variousoperations, such as operations for convert sensor data into atwo-dimensional rendering that corresponds to a gesture made with thetip in midair.

In some embodiments, the smart wand device may include a fingerprintscanner disposed on the portion of the elongate barrel housing sized tobe held by the user. The fingerprint scanner may be coupled to theprocessor and configured to detect fingerprint features of the user inresponse to the user engaging a finger on the fingerprint scanner whilethe user is holding the elongate barrel housing. In some embodiments,the smart wand device processor may be configured to receive datarepresenting the fingerprint features, and transmit the datarepresenting the fingerprint features, using the transceiver, to theremote computing device. In some embodiments, the smart wand deviceprocessor may be configured to receive an indication from the remotecomputing device that the data representing the fingerprint features wasaccepted for unlocking the remote computing device.

In some embodiments, the reference point may include information thatcorresponds to a nib of the smart wand device. In some embodiments, thesmart wand device of may include an orientation sensor that is securedto the elongate barrel housing and configured to determine changes in anorientation of the elongate barrel housing.

Various embodiments may be implemented in any of a variety of remotecomputing devices, an example of which is illustrated in FIG. 12. Forexample, the remote computing device 200 may include a processor 1202coupled to a touch screen controller 1204 and an internal memory 1206.The processor 1202 may be one or more multicore ICs designated forgeneral or specific processing tasks. The internal memory 1206 may bevolatile or non-volatile memory, and may be secure and/or encryptedmemory, or unsecure and/or unencrypted memory, or any combinationthereof.

The touch screen controller 1204 and the processor 1202 may also becoupled to a touch screen panel of the display 205, such as aresistive-sensing touch screen, capacitive-sensing touch screen,infrared sensing touch screen, etc. The processor 1202 may be coupled totwo or more radio signal transceivers 1208, 1209 and antennas 1210, 1211that enable communications via two or more cellular networks for sendingand receiving voice and data calls. The transceivers 1208, 1209 andantennas 1210, 1211 may be used with the above-mentioned circuitry toimplement the various wireless transmission modem stacks and interfaces.

The remote computing device 200 may include a peripheral deviceconnection interface 1218 coupled to the processor 1202. The peripheraldevice connection interface 1218 may be singularly configured to acceptone type of connection, or multiply configured to accept various typesof physical and communication connections, common or proprietary, suchas USB, FireWire, Thunderbolt, or PCIe. The peripheral device connectioninterface 1218 may also be coupled to a similarly configured peripheraldevice connection port (not shown). The peripheral device connectioninterface 1218 may be used to provide a wired connection between theremote computing device 200 and the smart wand device. The remotecomputing device 200 may also include speakers 1214 for providing audiooutputs. The remote computing device 200 may also include a housing1220, constructed of a plastic, metal, or a combination of materials,for containing all or some of the components discussed herein. Theremote computing device 200 may include a power source 1222 coupled tothe processor 1202, such as a disposable or rechargeable battery. Therechargeable battery may also be coupled to the peripheral deviceconnection port to receive a charging current from a source external tothe remote computing device 200.

The processors 150, 1202 in FIGS. 2 and 12 respectively, may be anyprogrammable microprocessor, microcomputer or multiple processor chip orchips that can be configured by software instructions (applications) toperform a variety of functions, including the functions of variousembodiments described above. In some devices, multiple processors may beprovided, such as one processor dedicated to wireless communicationfunctions and one processor dedicated to running other applications.Typically, software applications may be stored in the internal memory152, 1206 (referring again to FIGS. 2 and 12 respectively) before theyare accessed and loaded into the processors 150, 1202. Processors 150,1202 may include internal memory sufficient to store the applicationsoftware instructions. In many devices, the internal memory may be avolatile or nonvolatile memory, such as flash memory, or a mixture ofboth. For the purposes of this description, a general reference tomemory refers to memory accessible by the processors 150, 1202 includinginternal memory or removable memory plugged into the remote computingdevice and memory within the processor 150, 1202, themselves.

As used in this application, the phrase “direct wireless link” refers toany short-range direct wireless data communication connection betweentwo computing devices or within a personal area network. Unlike acellular telephone call, a direct wireless link does not include anintermediary wide area network (WAN) access point. For example, a directwireless link may use Bluetooth or Wi-Fi standards/protocols to linkdirectly two mobile communication devices.

The term “computing device” is used generically in this application torefer to any one or all of servers, personal computers, laptopcomputers, tablet computers, mobile devices, cellular telephones,smartbooks, ultrabooks, palm-top computers, personal data assistants(PDA's), wireless electronic mail receivers, multimedia Internet enabledcellular telephones, Global Positioning System (GPS) receivers, wirelessgaming controllers, and other similar electronic devices that include aprogrammable processor and circuitry for wirelessly sending or receivinginformation.

The terms “mobile device,” “wireless mobile device” and “mobilecomputing device” may be used interchangeably in this application, andrefer to any one or all of cellular telephones, smartphones, personal ormobile multimedia players, watches, wrist displays, medical devices,headsets, headphones, speakers, microphones, and/or any electronicdevice that includes circuitry for wirelessly sending and/or receivinginformation.

Terms “component,” “module,” “system,” “engine,” “generator,” “manager”and the like are intended to include a computer-related entity, such as,but not limited to, hardware, firmware, a combination of hardware andsoftware, software, or software in execution, which are configured toperform particular operations or functions. For example, a component maybe, but is not limited to, a process running on a processor, aprocessor, an object, an executable, a thread of execution, a program,and/or a computer. By way of illustration, both an application runningon a computing device and the computing device may be referred to as acomponent. One or more components may reside within a process and/orthread of execution and a component may be localized on one processor orcore and/or distributed between two or more processors or cores. Inaddition, these components may execute from various non-transitorycomputer readable media having various instructions and/or datastructures stored thereon. Components may communicate by way of localand/or remote processes, function or procedure calls, electronicsignals, data packets, memory read/writes, and other known network,computer, processor, and/or process related communication methodologies.

The term “Bluetooth®-enabled device” may be used in this application torefer to any electronic device that includes a radio frequency (RF)radio and a processor or circuitry for implementing the Bluetooth®protocol stack/interface. Bluetooth® is an open standard for short-rangeradio frequency (RF) communications. Details of the Bluetooth®standards, interfaces, and technology are set forth in Bluetooth®Special Interest Group (SIG) Specification of the Bluetooth® System CoreVersion 5.0 Dec. 6, 2016, which is herein incorporated by reference inits entirety.

Bluetooth® technology provides a secure way to connect and exchangeinformation between electronic devices (e.g., headphones, cellularphones, watches, laptops, remote controls, etc.). Bluetooth® requiresthat devices first establish a “trust relationship” before they areallowed to connect to one another. This is because many of the servicesoffered over Bluetooth® can expose private data or allow the connectingparty to control the connected device. A trust relationship may beestablished using a process called “pairing” in which a bond formedbetween the two devices. This bond enables the devices to communicatewith each other in the future without further authentication.

The pairing process may be triggered by a specific request to create abond (e.g., user explicitly requests to “add a Bluetooth® device”), ormay be triggered automatically (e.g., when connecting to a service). ABluetooth® device may automatically initiate the performance of thepairing operations each time the device is powered or moved within acertain distance of another Bluetooth® device. Pairing informationrelating to current and previously established pairings may be stored ina paired device list (PDL) in the memory of the Bluetooth® device. Thispairing information may include a name field, an address field, a linkkey field, and other similar fields (e.g., profile type, etc.) usefulfor authenticating the device and/or establishing a Bluetooth®communications link.

Bluetooth® communications may require establishing wireless personalarea networks (also referred to as “ad hoc” or “peer-to-peer” networks).These ad hoc networks are commonly called “piconets.” Each device maybelong to multiple piconets. Multiple interconnected piconets may becalled scatternets. A scatternet may be formed when a member of a firstpiconet elects to participate in a second piconet.

Any electronic device that includes a radio frequency (RF) radio and/orcircuitry implementing a short wave wireless protocol/interface is awireless-enabled device capable of communicating using short wavewireless technology. Such RF radios and circuitry are may be embedded insmall electronic devices (e.g., smart wand devices, etc.), allowingthese devices to communicate using wireless technology and replacing theneed for wires or wire based communications.

The foregoing method descriptions and the process flow diagrams areprovided merely as illustrative examples and are not intended to requireor imply that the steps of various embodiments must be performed in theorder presented. As will be appreciated by one of skill in the art theorder of steps in the foregoing embodiments may be performed in anyorder. Words such as “thereafter,” “then,” “next,” etc. are not intendedto limit the order of the steps; these words are simply used to guidethe reader through the description of the methods. Further, anyreference to claim elements in the singular, for example, using thearticles “a,” “an” or “the” is not to be construed as limiting theelement to the singular.

The various illustrative logical blocks, modules, circuits, andalgorithm steps described in connection with the embodiments disclosedherein may be implemented as electronic hardware, computer software, orcombinations of both. To clearly illustrate this interchangeability ofhardware and software, various illustrative components, blocks, modules,circuits, and steps have been described above generally in terms oftheir functionality. Whether such functionality is implemented ashardware or software depends upon the particular application and designconstraints imposed on the overall system. Skilled artisans mayimplement the described functionality in varying ways for eachparticular application, but such implementation decisions should not beinterpreted as causing a departure from the scope of the presentinvention.

The hardware used to implement the various illustrative logics, logicalblocks, modules, and circuits described in connection with the aspectsdisclosed herein may be implemented or performed with a general purposeprocessor, a digital signal processor (DSP), an application specificintegrated circuit (ASIC), a field programmable gate array (FPGA) orother programmable logic device, discrete gate or transistor logic,discrete hardware components, or any combination thereof designed toperform the functions described herein. A general-purpose processor maybe a microprocessor, but, in the alternative, the processor may be anyconventional processor, controller, microcontroller, or state machine. Aprocessor may also be implemented as a combination of computing devices,e.g., a combination of a DSP and a microprocessor, a plurality ofmicroprocessors, one or more microprocessors in conjunction with a DSPcore, or any other such configuration. Alternatively, some steps ormethods may be performed by circuitry that is specific to a givenfunction.

In one or more embodiments, the functions described may be implementedin hardware, software, firmware, or any combination thereof. Ifimplemented in software, the functions may be stored as one or moreinstructions or code on a non-transitory computer-readable medium ornon-transitory processor-readable medium. The steps of a method oralgorithm disclosed herein may be embodied in a processor-executablesoftware module or component that may reside on a non-transitorycomputer-readable or processor-readable storage medium. Non-transitorycomputer-readable or processor-readable storage media may be any storagemedia that may be accessed by a computer or a processor. By way ofexample but not limitation, such non-transitory computer-readable orprocessor-readable media may include RAM, ROM, EEPROM, FLASH memory,CD-ROM or other optical disk storage, magnetic disk storage or othermagnetic storage devices, or any other medium that may be used to storedesired program code in the form of instructions or data structures andthat may be accessed by a computer. Disk and disc, as used herein,includes compact disc (CD), laser disc, optical disc, digital versatiledisc (DVD), floppy disk, and blu-ray disc where disks usually reproducedata magnetically, while discs reproduce data optically with lasers.Combinations of the above are also included within the scope ofnon-transitory computer-readable and processor-readable media.Additionally, the operations of a method or algorithm may reside as oneor any combination or set of codes and/or instructions on anon-transitory processor-readable medium and/or computer-readablemedium, which may be incorporated into a computer program product.

The preceding description of the disclosed embodiments is provided toenable any person skilled in the art to make or use the presentinvention. Various modifications to these embodiments will be readilyapparent to those skilled in the art, and the generic principles definedherein may be applied to other embodiments without departing from thespirit or scope of the invention. Thus, the present invention is notintended to be limited to the embodiments shown herein but is to beaccorded the widest scope consistent with the following claims and theprinciples and novel features disclosed herein.

What is claimed is:
 1. A smart wand device, comprising: an elongatebarrel housing extending between a tip and an end cap, wherein a portionof the elongate barrel housing is sized to be held by a user; a positionsensor secured to the elongate barrel housing and configured todetermine changes in position of a reference point of the smart wanddevice; a transceiver; and a processor disposed within the elongatebarrel housing and coupled to at least the position sensor and thetransceiver, wherein the processor is configured withprocessor-executable instructions to perform operations comprising:receiving sensor data from the position sensor, wherein the sensor datameasures movements of a gesture made with the tip in midair by the userholding the elongate barrel housing, wherein the gesture is made withoutcontact between the tip and a surface; and transmitting the sensor data,using the transceiver, to a remote computing device for conversion intoa two-dimensional rendering corresponding to the gesture made with thetip in midair by the user.
 2. The smart wand device of claim 1, furthercomprising: a fingerprint scanner disposed on the portion of theelongate barrel housing sized to be held by the user, wherein thefingerprint scanner is coupled to the processor and configured to detectfingerprint features of the user in response to the user engaging afinger on the fingerprint scanner while the user is holding the elongatebarrel housing.
 3. The smart wand device of claim 2, wherein theprocessor is configured with processor-executable instructions to:receive data representing the fingerprint features; and transmit thedata representing the fingerprint features, using the transceiver, tothe remote computing device.
 4. The smart wand device of claim 3,wherein the processor is configured with processor-executableinstructions to: receive an indication from the remote computing devicethat the data representing the fingerprint features was accepted forunlocking the remote computing device.
 5. The smart wand device of claim1, wherein the reference point corresponds to a nib of the smart wanddevice.
 6. The smart wand device of claim 1, further comprising: anorientation sensor secured to the elongate barrel housing and configuredto determine changes in an orientation of the elongate barrel housing.7. A method of collecting data with a smart wand held in a hand of auser, comprising: receiving sensor data measuring a gesture made withthe smart wand in midair by the hand of the user, wherein the gesture ismade without contact between the smart wand and a surface other than thehand of the user; and transmitting the sensor data to a remote computingdevice for generating a two-dimensional rendering corresponding to thegesture made with the smart wand in midair by the hand of the user. 8.The method of claim 7, wherein the two-dimensional rendering includes atleast one drawn line or mark that matches a pattern of the gesture. 9.The method of claim 7, wherein the gesture includes shorthand used forgenerating the two-dimensional rendering.
 10. The method of claim 7,wherein the two-dimensional rendering includes at least one alphanumericcharacter.
 11. The method of claim 7, wherein generating thetwo-dimensional rendering uses a previous training sequence thatcorrelates the gesture to the two-dimensional rendering.
 12. The methodof claim 7, further comprising: converting the sensor data to renderingdata configured to generate a visual representation of the gesture madewith the smart wand on a display of the remote computing device, whereinthe sensor data transmitted to the remote computing device includes therendering data converted from the sensor data.
 13. The method of claim7, further comprising: receiving an authorization input corresponding tothe user contacting a first button on the smart wand; and transmittingan unlock activation message to the remote computing device in responseto receiving the authorization input, wherein the unlock activationmessage includes the authorization input for unlocking the remotecomputing device from a locked state.
 14. The method of claim 13,wherein reception by the remote computing device of the unlockactivation message does not require line-of-sight between the smart wandand the remote computing device.
 15. The method of claim 13, wherein thefirst button includes a fingerprint sensor and the authorization inputincludes fingerprint data collected by the fingerprint sensor on thesmart wand.
 16. The method of claim 13, wherein the authorization inputincludes a sequence of button presses with a particular timing sequencethat acts as an authentication code for unlocking the remote computingdevice from the locked state.
 17. The method of claim 7, furthercomprising: receiving an application launch input corresponding to theuser contacting an application launch button on the smart wand; andtransmitting an application activation message to the remote computingdevice in response to receiving the application launch input, whereinthe application activation message acts to activate an application onthe remote computing device.
 18. The method of claim 17, wherein theapplication activated by the application activation message receives: anaudio input from a microphone of the remote computing device; and avideo input from a camera of the remote computing device.
 19. Anon-transitory computer readable storage medium having stored thereonprocessor-executable software instructions configured to cause aprocessor to perform operations for collecting data with a smart wandheld in a hand of a user, the operations comprising: receiving sensordata measuring a gesture made with the smart wand in midair by the handof the user, wherein the gesture is made without contact between thesmart wand and a surface other than the hand of the user; andtransmitting the sensor data to a remote computing device for generatinga two-dimensional rendering corresponding to the gesture made with thesmart wand in midair by the hand of the user.
 20. The non-transitorycomputer readable storage medium of claim 19, wherein: the storedprocessor-executable software instructions are configured to cause theprocessor to perform operations further comprising: receiving anauthorization input corresponding to the user contacting a first buttonon the smart wand; and transmitting an unlock activation message to theremote computing device in response to receiving the authorizationinput; and the stored processor-executable software instructions areconfigured to cause the processor to perform operations such that: theunlock activation message includes the authorization input for unlockingthe remote computing device from a locked state; the reception by theremote computing device of the unlock activation message does notrequire line-of-sight between the smart wand and the remote computingdevice; the first button includes a fingerprint sensor; and theauthorization input includes fingerprint data collected by thefingerprint sensor on the smart wand.